Papers by Neha Anna John
Characterizing and Measuring Linguistic Dataset Drift (2023.acl-long)
Copied to clipboard
Tyler Chang, Kishaloy Halder, Neha Anna John, Yogarshi Vyas, Yassine Benajiba, Miguel Ballesteros, Dan Roth
| Challenge: | Existing metrics for dataset drift have not considered specific dimensions of linguistic drift that affect model performance. |
| Approach: | They propose three dimensions of linguistic dataset drift: vocabulary, structural, and semantic drift. |
| Outcome: | The proposed metrics are more effective than previous metrics at predicting out-of-domain model accuracies compared to popular fine-tuned embedding distances . |
A Weak Supervision Approach for Few-Shot Aspect Based Sentiment Analysis (2024.eacl-long)
Copied to clipboard
Robert Vacareanu, Siddharth Varia, Kishaloy Halder, Shuai Wang, Giovanni Paolini, Neha Anna John, Miguel Ballesteros, Smaranda Muresan
| Challenge: | Existing methods to improve few-shot performance in aspect-based sentiment analysis (ABSA) require complex interactions between the target and the polarity of the sentiment. |
| Approach: | They propose a pipeline approach to construct a noisy ABSA dataset and adapt it to the ABSA tasks. |
| Outcome: | The proposed model outperforms the state-of-the-art on the aspect extraction sentiment classification task and is capable of performing the harder aspect sentiment triplet extraction task. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Open Domain Question Answering with Conflicting Contexts (2025.findings-naacl)
Copied to clipboard
Siyi Liu, Qiang Ning, Kishaloy Halder, Zheng Qi, Wei Xiao, Phu Mon Htut, Yi Zhang, Neha Anna John, Bonan Min, Yassine Benajiba, Dan Roth
| Challenge: | Open domain question answering systems often rely on information retrieved from large collections of text to answer questions. |
| Approach: | They evaluate and benchmark three powerful Large Language Models with a dataset . they find that 25% of unambiguous open domain questions can lead to conflicting contexts . |
| Outcome: | The proposed model can't be used to answer questions with conflicting contexts . it can be fine tuned to provide richer information into the model's training . |
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views (2023.eacl-main)
Copied to clipboard
| Challenge: | Temporal concept drift is a problem of data changing over time. |
| Approach: | They benchmark 11 pretrained masked language models on a series of tests to evaluate temporal concept drift. |
| Outcome: | The proposed framework evaluates 11 pretrained masked language models on a series of tests . it aims to reveal how robust an MLM is over time and provide a signal in case it has become outdated . |
Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models (2025.findings-acl)
Copied to clipboard
Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, Yassine Benajiba
| Challenge: | LLaVA-7B demonstrated a decline in safety alignment ability on multi-modal inputs compared to its LLM backbone. |
| Approach: | They propose a method to recover alignment ability from LLM backbone while preserving functional capabilities of VLMs. |
| Outcome: | The proposed framework recovers alignment ability that is inherent in the LLM backbone with minimal impact on fluency and linguistic capabilities of pre-trained VLMs. |
Understanding and Improving Information Preservation in Prompt Compression for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled their successful application to a broad range of tasks. |
| Approach: | They propose a framework that allows for in-depth analysis of prompt compression methods. |
| Outcome: | The proposed framework analyzes state-of-the-art soft and hard compression methods . it shows that some fail to preserve key details from the original prompt, limiting performance on complex tasks. |
Towards Long Context Hallucination Detection (2025.findings-naacl)
Copied to clipboard
Siyi Liu, Kishaloy Halder, Zheng Qi, Wei Xiao, Nikolaos Pappas, Phu Mon Htut, Neha Anna John, Yassine Benajiba, Dan Roth
| Challenge: | Large language models are prone to contextual hallucination, generating information that is either unsubstantiated or contradictory to the given context. |
| Approach: | They propose a dataset specifically designed for long-context hallucination detection. |
| Outcome: | The proposed architecture outperforms existing models while providing faster inference. |